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SOCIAL  AWARENESS  AND  LEADER  INFLUENCE:  DEVELOPMENT  OF  CLASSROOM 
AND  WEB  BASED  LEARNING  INTERVENTIONS 

EXEUTIVE  SUMMARY 


Research  Requirements: 

In  the  current  operational  environment,  the  need  for  socially-aware  and  influential  leaders  at 
all  levels  in  the  military  has  become  increasingly  apparent.  However,  little  fonnal  training  exists 
to  help  new  Army  leaders  enhance  these  skills.  To  address  this  training  gap,  the  Ft.  Leavenworth 
Research  Unit  of  ARI  (ARI-FLRU)  initiated  research  in  May  of  2005  on  the  topic  of  “Social 
Awareness  and  Influence  on  Others.”  The  goal  of  this  initial  research  was  to  develop  a  training 
concept  that  would  accelerate  leader  development  through  exposure  to  structured  experiences  in 
self-awareness,  self-modulation,  and  techniques  to  enhance  leader  social  influence  supported  by 
real-time  measurement,  coaching,  and  feedback. 

In  support  of  this  goal,  an  extensive  literature  review  was  conducted  to  develop  and 
document  a  model  of  social  awareness  and  influence  based  on  an  analysis  of  relevant  research. 
Using  the  model  as  a  guide,  training  concepts  were  pilot  tested  to  address  linkages  in  the  model. 
The  results  from  this  effort  were  then  summarized  in  a  technical  report  that  provides  preliminary 
evidence  of  the  efficacy  of  the  social  awareness  and  influence  training  concepts  (Mueller- 
Hanson,  Swartout,  Morewitz,  Keil,  McGonigle,  Martin,  Parish,  &  Morath,  2007). 

As  described  in  this  report,  ARI-FLRU  initiated  a  follow-on  research  project  to  extend 
understanding  of  the  social  awareness  and  influence  process  and  to  provide  refined  tools  to 
support  Army  leader  development.  The  objectives  of  this  follow-on  work,  then,  were  to  develop 
a  more  comprehensive  and  detailed  program  of  instruction;  initiate  a  web-based  platform 
program  of  instruction  to  train  and  enhance  self-awareness,  self-regulation,  and  social  influence 
skills;  and  to  evaluate  the  program  and  provide  a  detailed  proof  of  concept  and  design  for  web 
implementation  as  a  program  of  instruction. 

Procedure: 

Based  on  input  from  Soldier  subject  matter  experts  and  the  behavioral  sciences  literature,  two 
major  training  programs  were  developed:  a  six-hour  classroom  course  and  a  parallel  web-based 
course.  Additionally,  four  assessments  were  created:  a  Social  Awareness  and  Influence  Skills 
Self-assessment  to  assess  self-perceptions  of  skills  in  these  areas,  a  knowledge  test  to  assess 
learning  in  the  course,  a  situational  judgment  test  (SJT)  to  assess  the  students’  ability  to  apply 
social  insight  and  judgment  to  interpersonal  situations,  and  a  web-based  role-play  to  assess  the 
students’  ability  to  apply  influence  tactics.  These  courses  and  assessments  were  then 
subsequently  tested  in  two  pilot  samples. 

The  first  sample  consisted  of  48  Soldiers  (18  officers  and  30  enlisted  personnel)  from 
location  1  who  participated  in  the  classroom  course.  Classroom  assessments  included  the  self- 
assessment  and  SJT.  Half  the  participants  completed  the  SJT  before  the  training,  and  half 
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completed  the  SJT  at  the  end  of  the  training.  The  second  sample  consisted  of  41  officers  at 
location  2  who  participated  in  the  web-based  course;  this  sample  participated  in  this  project  as 
part  of  their  training.  The  second  sample  was  further  divided  into  two  groups:  a  control  group 
and  a  training  group.  The  control  group  was  asked  to  complete  all  four  assessments  followed  by 
the  training  course,  and  the  training  group  completed  the  training  course  first,  followed  by  the 
assessments.  Evaluation  data  were  collected  at  two  levels  to  assess  the  effectiveness  of  the 
training:  participant  reactions  (level  1)  and  acquisition  of  new  knowledge  and  skills  (level  2). 

The  level  1  evaluation  was  conducted  using  a  survey  administered  at  the  end  of  each  training 
session.  The  level  2  evaluation  consisted  of  the  scores  on  the  applicable  assessments. 

Findings: 

Results  of  the  participant  reactions  questionnaire  indicate  that  the  training  programs  in  both 
the  classroom  and  web-based  formats  were  well  received  by  the  target  audience  and  that 
participants  gained  new  knowledge  as  a  result  of  the  course.  In  the  web-based  instruction  sample, 
individuals  in  the  training  condition  scored  significantly  higher  on  the  knowledge  test  than 
individuals  in  the  control  group.  However,  differences  in  situational  judgment  test  scores  (for 
either  the  classroom  or  the  web-based  samples)  and  role-play  scores  were  not  significant 
between  the  two  conditions  in  the  web-based  instruction  sample.  Given  that  significant 
improvements  were  observed  in  role-play  perfonnance  before  and  after  the  training  in  earlier 
classroom  pilot  tests  (Mueller-Hanson,  et  al.,  2007),  one  possible  conclusion  from  this  finding  is 
that  a  short  online  training  program  is  not  sufficient  for  building  new  skills,  but  it  is  useful  in 
developing  knowledge.  Therefore,  web-based  training  such  as  the  program  developed  for  this 
project  may  be  most  useful  as  pre-work  to  prepare  students  for  more  intensive,  scenario-based 
training  in  a  classroom  setting,  which  is  a  better  forum  for  building  skills. 

Results  also  indicated  promise  for  the  piloted  measures.  The  self-report  measure  was  a 
reliable  indicator  of  perceived  social  awareness  and  influence  skills,  and  this  measure  was 
correlated  with  performance  on  the  situational  judgment  test.  In  the  web-based  instruction 
sample,  perfonnance  on  the  situational  judgment  test  correlated  with  performance  on  the 
automated  role-play.  As  these  measures  were  prototype  instruments,  more  research  is  needed  to 
validate  these  results  with  external  performance. 

Utilization  and  Dissemination  of  Results: 

The  outcomes  of  this  effort  include  a  fully  developed  classroom-based  POI  and  a  companion 
prototype  web-based  POI  that  are  suitable  for  first-line  Army  leaders  (E4  to  E6  and  01  to  03). 
While  the  classroom  course  has  been  thoroughly  tested  and  is  ready  for  dissemination,  the  web- 
based  training  program  and  assessments  are  prototypes  and  could  benefit  from  further 
enhancements,  as  suggested  via  feedback  from  Soldiers.  Specifically,  the  training  program  could 
be  enhanced  by  including  more  activities,  more  animation,  and  more  audio.  The  role-play 
assessment  could  benefit  from  including  a  coach  avatar  that  would  provide  guidance  and 
feedback  throughout  the  assessment  and  a  debriefing  at  the  end  so  that  they  received  some 
insight  into  the  Mukhtar’s  perspective.  Finally,  the  situational  judgment  test  could  benefit  by 
enhancing  the  quality  of  the  video. 
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Introduction 


Overview 

In  the  current  operational  environment,  the  need  for  socially-aware  and  influential  leaders  at 
all  levels  in  the  military  has  become  increasingly  apparent.  During  Operation  Enduring  Freedom, 
military  leaders  found  they  were  fighting  a  different  kind  of  war  in  an  unfamiliar  culture  with  a 
dispersed,  yet  tenacious,  enemy.  Subsequent  efforts  in  Iraq,  increased  operational  tempo,  a  high 
degree  of  uncertainty,  and  the  need  to  constantly  shift  tactics  and  approaches  have  all  contributed 
to  an  environment  in  which  leader  social  awareness,  adaptability,  and  influence  skills  are 
required  for  mission  success. 

The  need  for  additional  leader  social  awareness,  adaptability,  influence,  and  related  skills  has 
been  well-documented.  For  example,  the  Anny  Leadership  Development  and  Training  Panel 
(ATLDP;  Department  of  the  Army,  2001)  concluded  that  adaptability  and  self-awareness  are 
critical  skills  for  leaders.  Similarly,  Horey,  Fallesen,  Morath,  Cronin,  Cassella,  Franks,  and 
Smith  (2004)  included  social  influence  (both  within  and  beyond  the  chain  of  command)  and 
awareness  of  self  as  components  of  their  Army  leader  competency  model.  Wong,  Gerrars,  Kidd, 
Pricone,  and  Swengros  (2003)  also  describe  six  metacompetencies  that  are  necessary  for 
strategic  leadership  in  the  Army’s  future,  four  of  which  are  related  to  the  concepts  of  self- 
awareness  and  adaptability  (and  by  extension  social  influence):  identity,  mental  agility,  cross- 
cultural  savvy,  and  interpersonal  maturity.  Additionally,  the  importance  of  influence  in  Army 
doctrine  is  emphasized  in  the  Army  Leadership  field  manual  (FM  6-22,  p.1-2),  which  positions 
influence  firmly  at  the  foundation  of  leadership,  indicating  that,  “Leadership  is  the  process  of 
influencing  people  by  providing  purpose,  direction,  and  motivation  while  operating  to 
accomplish  the  mission  and  improving  the  organization.”  However,  despite  the  increased  need 
for  socially  aware  leaders  who  are  skilled  in  influencing  others,  little  fonnal  training  exists  to 
ensure  leaders  are  properly  trained  in  these  skill  sets. 

To  address  this  training  gap,  the  Ft.  Leavenworth  Research  Unit  of  ARI  (ARI-FLRU) 
initiated  research  in  May  of  2005  on  the  topic  of  “Social  Awareness  and  Influence  on  Others.” 
The  goal  of  this  initial  research  was  to  develop  a  training  concept  that  would  accelerate  leader 
development  through  exposure  to  structured  experiences  in  self-awareness,  self-modulation,  and 
techniques  to  enhance  leader  social  influence  supported  by  real-time  measurement,  coaching, 
and  feedback. 

PDRI,  with  sub-contractor  IGF  International  led  efforts  to  investigate  the  social  awareness 
and  influence  process.  Specifically,  the  research  team  conducted  an  extensive  literature  review  to 
develop  and  document  a  model  of  social  awareness  and  influence  based  on  an  analysis  of 
relevant  research.  The  research  team  developed  a  methodology  to  pilot  test  the  training  concepts 
for  the  different  linkages  in  the  model.  The  results  from  this  effort  were  then  summarized  in  a 
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technical  report  that  provides  preliminary  evidence  of  the  efficacy  of  the  social  awareness  and 
influence  training  concepts  (Mueller-Hanson,  Swartout,  Morewitz  et  ah,  2007). 

The  follow-on  research  project  described  in  this  report  was  initiated  by  ARI-FLRU  to  extend 
understanding  of  the  social  awareness  and  influence  process  and  to  provide  refined  tools  to 
support  Army  leader  development.  The  specific  objectives  of  this  follow-on  work  were  to  1) 
develop  a  more  comprehensive  and  detailed  program  of  instruction,  2)  initiate  a  web-based 
platform  program  of  instruction  to  train  and  enhance  social  awareness,  self-regulation,  and  social 
influence  skills,  and  3)  evaluate  the  program  and  provide  a  detailed  proof  of  concept  and  design 
for  web  implementation  as  a  program  of  instruction. 

This  report  details  the  results  of  this  follow-on  work.  It  is  organized  into  four  main  sections. 
First,  the  introduction  presents  a  summary  of  the  prior  research.  Second,  the  methodology  section 
documents  how  the  project  was  carried  out.  Third,  the  results  section  reports  data  from  the  pilot 
tests.  Finally,  the  discussion  section  presents  implications  of  these  results. 

Background 

The  initial  project  (Mueller-Hanson,  et  ah,  2007)  led  to  the  development  of  a  model  of  social 
awareness  and  influence  (see  Figure  1)  including  the  factors  hypothesized  to  impact  this  process. 
The  model  defines  social  awareness  as  the  ability  of  an  individual  to  correctly  recognize  how  he 
or  she  is  perceived  and  the  ability  to  alter  one’s  behavior  accordingly.  As  such,  this  concept  is 
closely  related  to  the  concept  of  interpersonal  adaptability  (Pulakos,  Arad,  Donovan,  & 
Plamondon,  2000).  White  et  ah,  2005  defined  interpersonal  adaptability  as  “adjusting  what  one 
says  and  does  to  make  interactions  with  other  people  run  more  smoothly  and  effectively.  This 
includes  trying  to  understand  the  needs  and  motives  of  other  people  -  especially  people  in  other 
cultures”  (p.3).  According  to  this  model  an  influence  attempt  occurs  in  three  phases:  the  planning 
phase,  the  interaction  phase,  and  the  in-the-moment  awareness  phase. 

During  the  Planning  Phase  the  influencer  identifies  his  or  her  influence  goal,  evaluates 
the  situation,  and  decides  upon  an  initial  influence  strategy.  Planning  may  occur  prior  to  the 
interaction  or  at  the  point  of  first  meeting.  The  Interaction  Phase  refers  to  the  actual  interaction 
between  the  influencer  and  the  target  of  their  influence  attempt.  During  this  phase,  the  influencer 
uses  one  or  more  tactics  to  influence  the  target.  The  target  interprets  the  meaning  of  these  tactics 
and  has  a  verbal  and  nonverbal  reaction  to  the  influence  attempt.  During  the  final  phase,  the  In- 
the -Moment  Awareness  Phase,  the  influencer  perceives  the  target’s  reaction,  analyzes  it  to 
interpret  its  meaning,  and  evaluates  that  interpretation  against  the  original  influence  goal.  If  the 
influencer  concludes  that  the  goal  is  met,  the  influence  attempt  may  conclude 
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Figure  1.  A  Model  of  Social  Awareness  and  Influence 

If  the  goal  has  not  been  met,  the  influencer  will  need  to  adjust  his  or  her  approach  to  try  a 
different  influence  tactic.  This  process  may  continue  in  an  iterative  fashion  until  an  agreement  is 
reached  or  until  the  parties  are  at  an  impasse. 

Using  the  model  described  above,  a  group  of  subject  matter  experts  was  asked  to  rate  factors 
that  might  affect  this  model  in  terms  of  their  likely  impact  and  suitability  as  training  topics.  The 
results  of  the  expert  rating  task  revealed  that  experience  (both  as  a  leader  and  with  the  target), 
knowledge  of  non-verbal  behavior  and  cultural  norms,  communication/persuasion  skills,  political 
skills,  social  intelligence,  metacognition,  and  self-awareness  were  the  factors  that  were  most 
likely  to  have  a  high  impact  on  the  social  awareness  and  influence  process  and  that  could 
potentially  be  improved  through  training.  Consequently,  several  training  concepts  were 
identified:  knowledge  of  the  social  awareness  and  influence  process,  knowledge  of/skill  in 
reading  verbal  and  non-verbal  cues  of  others,  and  knowledge  of/skill  in  using  influence  tactics. 
Using  a  combination  of  commercially  available  tools  and  customized  content,  a  six-hour  social 
awareness  and  influence  training  program  was  developed.  This  program  was  then  pilot-tested 
with  72  Company  Grade  Officers  and  Non  Commissioned  Officers  (NCOs). 

The  pilot  investigation  served  a  dual  purpose.  First,  the  pilot  tested  the  various  training 
concepts  to  determine  their  suitability  for  ongoing  use.  Second,  the  pilot  was  useful  in  making 
some  initial  determinations  about  whether  social  awareness  and  influence  skills  could  be 
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improved  through  training.  Evaluation  data  were  collected  at  two  levels  to  assess  the 
effectiveness  of  the  training:  participant  reactions  (level  1)  and  acquisition  of  new  knowledge 
and  skills  (level  2).  The  level  1  evaluation  was  conducted  using  a  survey  administered  at  the  end 
of  each  training  session.  A  pre-post  test  design  was  used  to  conduct  the  level  2  evaluation. 
Participants  were  assessed  via  a  test  of  nonverbal  sensitivity  and  a  role-play  exercise  both  before 
and  after  the  training.  These  assessments  evaluated  students  on  their  ability  to  read  verbal  and 
non-verbal  cues  and  on  adjusting  their  own  behavior  to  more  effectively  influence  others. 

Results  of  the  participant  reactions  questionnaire  indicated  that  the  training  program  was  well 
received  by  the  target  audience.  Participants  reported  that  the  role-plays  and  instruction  on 
influence  tactics  were  the  most  useful  components  of  the  course.  Results  of  the  pre-post  tests 
provided  evidence  that  training  was  successful  in  helping  participants  improve  their  skills  in  the 
following  areas: 

♦  Clearly  communicating  an  influence  goal, 

♦  Adjusting  behavior  in  response  to  a  target’s  reactions,  and 

♦  Influencing  others  for  compliance  and  commitment 

The  results  of  this  initial  research  made  three  significant  contributions.  First,  based  on  a 
review  of  the  literature,  a  viable  model  of  in-the -moment  social  awareness  and  influence  was 
developed.  Second,  this  research  provided  support  for  the  notion  that  social  awareness  and 
influence  are  skills  that  can  be  enhanced  through  training.  A  third  contribution  of  this  project  is 
the  development  of  customized  behavioral  observation  measures  to  evaluate  role-play 
performance,  which  was  useful  in  both  student  assessment  and  training. 

Outline  of  Current  Project 

The  initial  results  (Mueller-Hanson,  et  ah,  2007)  suggested  that  the  classroom  program  of 
instruction  (POI)  was  useful  in  developing  social  awareness  and  influence  skills.  However,  these 
results  also  suggested  that  there  were  several  ways  in  which  the  course  could  be  improved, 
including: 

♦  Revise  organization  of  the  POI  to  better  follow  the  Model  of  Social  Awareness  and 
Influence 

♦  Enhance  role-plays  to  include  examples  from  SMEs  and  examples  appropriate  to  NCOs 
and  officers 

♦  Include  additional  case  studies  and  examples 

♦  Revise  influence  tactics  exercises  to  focus  on  military  examples 

♦  Eliminate  some  of  the  commercial  measures  used  in  the  course  (i.e.,  the  PONS  and 
SETT),  as  they  have  limited  utility  in  this  setting  and  were  not  well-received  by  the  target 
audience 

♦  Standardize  the  assessments  -  perhaps  with  the  assistance  of  technology 
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Additionally,  ARI-FLRU  was  interested  in  developing  a  prototype  web-based  version  of  the 
POI.  Web-based  training  (WBT)  offers  the  promise  of  delivering  learning  interventions  to  a 
distributed  workforce  in  a  cost-effective  manner.  The  U.S.  Army  has  recently  made  computer- 
based  learning  a  priority  because  of  the  need  to  deliver  training  to  troops  wherever  they  are 
deployed.  Computer-based  learning,  or  e-learning,  may  offer  other  benefits  as  well  such  as 
increases  in  learner  engagement  and  motivation.  In  fact,  it  has  been  argued  that  young  adults, 
members  of  the  tech-savvy  generation,  prefer  the  interactivity  offered  by  computer-based 
training  and  are  “turned  off’  by  traditional  training  methods  (Prensky,  2001). 

Despite  the  appeal  of  e-learning,  there  is  no  consensus  on  the  extent  to  which  technology  can 
be  used  instead  of  traditional  classroom  training.  For  example,  a  recent  review  conducted  by  the 
U.S.  Navy  urged  caution  in  the  use  of  instructional  games  (Hays,  2005).  According  to  this 
review,  evidence  supporting  the  effectiveness  of  computer-based  instructional  games  is 
fragmented.  While  e-learning  may  be  ideal  for  acquiring  declarative  knowledge,  it  is  not  clear 
whether  it  is  useful  in  developing  interpersonal  and  leadership  skills.  Moreover,  unless  the 
training  employs  learning  checks  and  assessments,  it  may  be  difficult  to  determine  the  extent  to 
which  trainees  are  attending  to  the  material.  Despite  these  barriers,  the  likely  long-tenn  cost 
effectiveness  and  logistical  benefits  to  e-learning  make  it  important  to  explore  its  potential  for 
training  a  wide  variety  of  skills. 

With  these  caveats  in  mind,  the  purpose  of  this  effort  was  twofold.  First,  the  effort  sought  to 
revise  and  pilot  test  the  classroom  POI.  Second,  the  effort  was  to  develop  and  pilot  test  a 
companion  prototype  web-based  POI  and  associated  assessments  to  determine  their  suitability 
for  ongoing  use.  A  key  component  of  this  research  was  to  explore  whether  technology-enabled 
training  would  have  a  positive  impact  on  developing  leader  social  awareness  and  influence  skills 
and  whether  the  benefits  of  WBT  offset  the  costs  of  less  interaction.  The  methodology  used  to 
develop  and  pilot  test  the  enhanced  classroom  POI  and  the  prototype  web-based  POI  is  presented 
in  the  next  section. 

Method 


Revisions  to  the  Classroom  Course 

As  a  starting  point  for  developing  the  full  training  POI,  it  was  necessary  to  develop  a 
measurement  model  based  on  the  theoretical  model  of  Social  Awareness  and  Influence  (see 
Figure  1)  and  research  conducted  in  the  initial  project  (Mueller-Hanson  et  ah,  2007).  This  model 
draws  from  the  principles  of  Evidence-Centered  Design  (Mislevy,  Steinberg,  Almond,  &  Lukas, 
2006),  in  which  the  assessment  development  process  includes  an  articulation  of  what  is  assessed, 
behaviors  that  indicate  proficiency  of  what  is  assessed,  and  the  knowledge  and  skills  that  relate 
to  these  behaviors.  The  measurement  model  includes  the  behavioral  definitions  of  each  major 
phase  of  the  social  awareness  and  influence  process,  behaviorally-based  criteria  that  can  be  used 
to  assess  success  in  each  phase,  and  the  associated  knowledge  and  skills  that  are  believed  to 
predict  these  criteria.  Developing  this  model  was  critical  to  the  training  development  process 
because  it  helped  in  identifying  specific  learning  objectives  for  the  course.  This  model  is 
presented  in  Table  1. 
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Table  1.  Social  Awareness  and  Influence  Measurement  Model 


Behavioral  Definition 

Criteria  for  Success 

Associated 
Knowledge  and 
Skills 

Planning  Phase 

♦  The  influence  goal  is  explicitly 
defined 

♦  The  influencer  scans  the  current 
environment  to  identify 
important  aspects  to  consider 
when  planning  how  he  or  she 
will  influence  the  target 

♦  The  influencer  plans  an  overall 
approach  or  strategy  for  the 
influence  attempt 

♦  The  goal  is  clearly  stated, 
includes  a  measurable 
outcome,  and  is  clearly  linked 
to  the  problem  (i.e.,  it  will 
solve  the  issue  at  hand) 

♦  Important  factors  (i.e.,  the 
factors  most  likely  to  impact 
the  influence  attempt)  are 
identified  and  the  potential 
impact  of  these  factors  is 
assessed 

♦  The  plan  incorporates 
important  factors  and  is 
designed  to  achieve  the  goal 

♦  Situational 

awareness 

♦  Cultural 

awareness 

♦  Self-awareness 

♦  Past  experience 
with/knowledge 
of  target 

Interaction  Phase 

♦  The  influencer  attempts  to 
accomplish  his  or  her  goal  by 
using  one  or  more  influence 
tactics  (the  actual  behaviors 
demonstrated  by  influencer 
during  the  interaction  with  the 
target) 

♦  The  target  perceives  and 
interprets  the  influencer’ s 
message 

♦  The  target  provides  a  reaction  to 
the  influence  attempt 

♦  Influence  tactics  are 
appropriate  to  the  situation 

♦  Tactics  contribute  to  meeting 
the  goal 

♦  Knowledge  of 
influence  tactics 

♦  Communication/ 
persuasion  skills 
and  experience 
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Behavioral  Definition 

Criteria  for  Success 

Associated 
Knowledge  and 
Skills 

In-the-Moment  Awareness  Phase  (Perceiving  and  Analyzing  Steps) 

♦  The  influencer  perceives  the 
target’s  response  and  analyzes  it 
to  determine  its  meaning 

♦  The  influencer  correctly 
identifies  the  target’s  reaction 
(i.e.,  influencer  and  target 
have  a  shared  view  of  what 
the  target  is  feeling) 

♦  The  influencer  correctly 
interprets  the  meaning  of  the 
target’s  reaction  (i.e.,  the 
influencer  and  target  both 
have  a  shared  understanding 
of  the  beliefs  and  motives  that 
underlie  the  target’s  reaction) 

♦  Knowledge  of 
verbal  and 
nonverbal  cues 

♦  Past  experience 
with/knowledge 
of  the  target 

♦  Cultural 

awareness 

♦  Social  perception 

In-the-Moment  Awareness  Phase  (Evaluating  Step) 

♦  The  influencer  compares  his/her 
interpretation  of  the  target’s 
reaction  with  the  initial 
influence  goal 

♦  If  the  influencer  detennines  that 
the  influence  goal  has  been  met, 
the  interaction  may  conclude 

♦  If  the  influencer  detennines  that 
the  goal  has  not  been  met,  he  or 
she  may  try  different  influence 
tactics  in  an  attempt  to  be  more 
effective 

♦  The  influencer  makes  an 
appropriate  choice  given  the 
situation  (e.g.,  concludes 
interaction  if  goal  is  met,  tries 
a  different  approach  if  the 
goal  is  not  met) 

♦  Emotional 
regulation 

♦  Metacognition 

In-the-Moment  Awareness  Phase  (Adjusting  Behavior  Step) 

♦  Using  the  information  from  the 
previous  steps,  the  influencer 
changes  the  way  s/he  interacts 
with  the  target  to  be  more 
effective 

♦  The  influencer  continues  to 
assess  the  target’s  reaction  and 
make  additional  adjustments  as 
needed 

♦  In  response  to  a  sub-optimal 
reaction  from  the  target,  the 
influencer  uses  a  different 
approach 

♦  The  new  approach  is  more 
effective  than  the  old 
approach  (i.e.,  brings  the 
influencer  closer  to  achieving 
his/her  goal) 

♦  Self-awareness 

♦  Self-regulation 
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Behavioral  Definition 

Criteria  for  Success 

Associated 
Knowledge  and 
Skills 

Influence  Outcome 

♦  Commitment:  Internal 

♦  Target’s  stated  compliance  or 

♦  Social  awareness 

agreement  to  the  request  and 

rejection  to  the  request 

skills 

persistence  in  carrying  it  out 

♦  Compliance:  Outward 

agreement  to  the  request  but  no 

♦  Target’s  actual  behavior  in 
following  through  to  carry  out 
the  request 

♦  Influence  skills 

indication  of  long-term 
commitment 

♦  Target’s  stated  commitment  to 
the  request 

♦  Resistance:  Refusing,  arguing, 
delaying,  or  otherwise  avoiding 
completing  the  request 

♦  The  manner  in  which  the 
target  carries  out  the  request 
(e.g.,  target’s  description  of 
the  request  to  others;  whether 
the  target  overcomes  obstacles 
to  ensure  the  request  is  carried 
out  or  finds  excuses  to  not 
complete  the  task) 

A  second  priority  in  revising  the  training  program  was  to  gather  additional  examples  for  the 
revised  POI.  To  this  end,  a  series  of  focus  groups  were  conducted  at  several  different  posts  (total 
N=94)  to  gather  critical  incidents  and  examples  that  could  be  used  to  enhance  the  role-plays  and 
vignettes  included  in  the  course.  Focus  group  participants  included  enlisted  ranks  ranging  from 
Staff  Sergeant  through  Master  Sergeant  and  officer  ranks  ranging  from  Second  Lieutenant  to 
Captain.  Group  sizes  ranged  from  8-12  individuals.  In  these  sessions,  participants  first  took  a 
short  version  of  the  original  Social  Awareness  and  Influence  course.  Then  they  were  asked  to 
provide  examples  from  their  own  experience  of  times  when  they  had  to  influence  peers, 
supervisors,  subordinates,  host-nation  personnel,  and  others  outside  their  chain  of  command. 
Participants  were  told  that  these  examples  would  be  used  to  refine  the  course  materials  in  the 
future.  Additionally,  Soldiers  were  asked  to  review  and  comment  on  the  course  content, 
including  existing  role-plays  and  to  make  suggestions  for  improvements. 

Using  the  measurement  model  as  a  guide,  the  course  objectives  were  refined  to  incorporate 
the  recommendations  identified  from  the  focus  groups  and  the  Phase  I  research.  The  course 
objectives  for  the  final  course  are  presented  in  Table  2.  Using  these  objectives  as  a  guide,  the 
course  contents  were  then  revised  accordingly. 
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Table  2.  Course  Learning  Objectives 


Terminal  Learning  Objective _ 

♦  Condition:  Given  an  influence  objective  and  a  role-play  scenario 

♦  Action:  Persuade  a  target  to  agree  to  carry  out  your  request 

♦  Standard:  Success  will  be  assessed  through  satisfactory  ratings  on  a  behavioral  observation 

checklist _ 

Enabling  Learning  Objectives _ 

♦  Introduction: 

Describe  the  relationship  between  influence  and  leadership 

Provide  an  example  of  how  influence  skills  can  affect  leadership  effectiveness 

Define  the  term  “social  awareness” 

Describe  the  relationship  between  social  awareness  and  influence 
Describe  the  three  types  of  outcomes  possible  in  an  influence  attempt 

♦  Pre-interaction  Phase: 

Identify  three  types  of  influence  goals 

Describe  three  environmental  factors  that  should  be  considered  when  planning  an 
influence  attempt  and  how  these  factors  impact  goal  selection  and  influence  strategy 
Describe  the  seven  power  bases  and  how  these  might  impact  the  selection  of  an  influence 
strategy 

♦  Interaction  Phase 

Given  a  list  of  influence  tactics  and  their  definitions,  describe  when  each  tactic  is 

appropriate  to  use  and  the  potential  benefits  and  drawbacks  of  each 

Given  a  list  of  influence  tactics  and  a  discussion  topic,  demonstrate  at  least  six  separate 

tactics 

♦  In-the-moment  Social  Awareness  Phase 

Describe  the  basic  verbal  and  nonverbal  cues  associated  with  the  following  emotions 
(happiness,  sadness,  anger,  fear,  surprise,  disgust,  and  contempt) 

Describe  the  fundamental  attribution  error  and  how  this  error  impacts  the  accuracy  of 
interpretations  of  other’s  verbal  and  nonverbal  cues 

Describe  how  cultural  biases  can  impact  judgment  about  others  and  how  these  biases  can 
be  mitigated 

In  the  context  of  an  influence  attempt,  demonstrate  four  techniques  for  modifying 
_ behavior  in  response  to  negative  reactions  from  a  target _ 
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An  outline  of  the  course  contents  is  presented  in  Table  3.  The  course  materials  are  available 
as  an  ARI  Research  Product  (Mueller-Hanson,  et  ah,  2008).  Course  materials  include  lecture 
slides  and  notes,  student  materials,  and  exercise  materials.  Exercises  include  role-plays,  video 
case  studies,  and  other  activities. 

Table  3.  Course  Outline 


Activities 

Time 

Sample 

Agenda 

I.  Course  Overview 

A.  Introduction/Welcome/  Overview  of  Session 

10  minutes 

0900-0910 

II.  Module  1:  The  Importance  of  Social  Awareness  and 

Influence 

20  minutes 

0910-0930 

TIT.  Module  2:  Planning  Phase 

A.  Influence  Goals 

5  minutes 

0930- 0935 

B.  Evaluation  of  the  Situation 

15  minutes 

0935- 0950 

•  Practice  Case  Study:  Clean-up  Crew 

BREAK 

10  minutes 

0950-  1000 

C.  Cultural  Considerations  (Values  Scale) 

20  minutes 

1000- 1020 

E.  Selecting  an  Influence  Strategy 

5  minutes 

1020- 1025 

F.  Practice  Case  Study 

10  minutes 

1025- 1035 

IV.  Module  3  -  Interaction  Phase 

A.  Influence  Tactics 

•  Rational  Persuasion-  Apprising 

15  minutes 

1035-  1050 

BREAK 

10  minutes 

1050- 1100 

•  Practice  Activity 

15  minutes 

1100-  1115 

•  Appeal  to  Higher  Authority  -  Blocking 

15  minutes 

1115-  1130 

•  Practice  Activity 

15  minutes 

1130- 1145 

LUNCH 

1  hr  15  min 

1145- 1300 

B.  Target’s  Interpretation  and  Reaction 

30  minutes 

1300-  1330 

•  Case  Study  Example  of  Fundamental 

Attribution  Error 

V.  Module  4  -  In-the-moment  Awareness  Phase 

A.  Perceiving,  Analyzing,  and  Evaluating  Social 

Information 

20  minutes 

1330-  1350 

BREAK 

10  minutes 

1350- 1400 

B.  Case  Study 

20  minutes 

1400- 1420 

C.  Summary 

10  minutes 

1420-  1430 

D.  Role-play  Exercise  (Note:  break  mid  way  through) 

45  minutes 

1430-  1515 

VI.  End  of  Course  Assessments 

A.  Assessments:  SJT  /  Knowledge  Test 

40  minutes 

1515- 1555 

B.  Course  Evaluation 

5  minutes 

1555-  1600 
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Development  of  the  WBT  Prototype  Course 

In  developing  the  WBT  prototype,  we  took  a  medium-tech  approach  rather  than  a  high-tech 
approach.  Following  the  recommendations  of  Clark  and  Mayer  (2008),  course  content  is 
presented  in  a  fashion  that  emphasizes  the  learning  objectives  and  de-emphasizes  extraneous 
content  that  is  provided  for  entertainment  only  and  that  may  distract  the  learners  from  the 
purpose  of  the  lesson.  The  WBT  development  process  entailed  the  following  steps: 

Define  requirements.  As  the  target  audience  for  this  training  was  unknown  at  the  time  of 
development,  the  requirements  had  to  remain  open-ended  to  fit  a  wide  array  of  users  with 
potentially  different  processor  speeds,  firewalls  and  other  security  measures,  screen  sizes, 
and  connection  speeds.  Therefore,  file  sizes  were  kept  small  to  accommodate  slow 
connection  speeds.  Additionally,  because  data  needed  to  be  captured  to  evaluate  the  training, 
a  web-based  platform  was  chosen  over  a  CD-ROM  format  because  it  would  be  easier  to 
capture  the  data  with  the  course  contents  integrated  with  a  learning  management  system 
(LMS). 

Storyboard  content.  During  the  storyboarding  process,  the  classroom  contents  were 
transferred  into  screenshots  of  the  text,  graphics,  and  pictures  that  would  appear  on  each 
screen.  All  photos  presented  in  the  WBT  program  are  courtesy  of  the  U.S.  Army,  and  many 
were  taken  from  the  U.S.  Army  website. 

Program  content  using  a  rapid  prototyping  approach.  All  content  with  the  exception  of 
the  role-play  was  developed  using  Flash  CS3  and  published  for  Flash  Player  8.  The  role-play 
was  developed  using  Adobe  Captivate.  The  content  was  integrated  into  Moodle,  an  open- 
source  LMS.  Moodle  was  chosen  because  of  its  flexibility,  ease  of  use,  ability  to  capture 
data,  and  ability  to  bookmark  pages  so  that  users  could  exit  the  program  at  any  time  and 
return  where  they  left  off.  After  each  module  was  storyboarded,  it  was  sent  to  the  web-design 
team  for  programming.  After  the  content  was  programmed,  it  was  sent  back  to  the 
instructional  design  team  for  review  and  testing.  Based  on  this  review,  changes  were  made  as 
necessary  and  the  content  was  integrated  into  Moodle  for  further  testing.  This  approach 
allowed  for  rapid  development  of  the  content. 

Pilot  test  content.  Initial  pilot  tests  were  conducted  with  five  Industrial/Organizational 
psychologists  and  Instructional  Systems  Design  (ISD)  specialists  who  had  not  participated  in 
the  training  development  process.  Revisions  were  made  based  on  the  results  of  these  initial 
tests.  Pilot  testing  with  Soldiers  is  described  in  the  next  section. 

Development  of  the  Measures 

The  following  measures  were  developed  as  assessments  to  be  used  in  either  the  classroom 
course  and/or  the  WBT  course: 

The  Social  Awareness  and  Influence  Skills  Self-assessment 
A  knowledge  test 
A  situational  judgment  test  (SJT) 

A  web-based  role-play 

A  description  of  each  of  these  measures  and  the  process  used  to  create  them  is  presented  below. 
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Social  Awareness  and  Influence  Skills  Self-assessment.  The  Social  Awareness  and 
Influence  Skills  Self-assessment  is  a  20  item  self-report  measure  of  one’s  social  awareness  and 
influence  skills.  A  paper  version  of  this  assessment  is  available  for  the  classroom  course  and  a 
web-enabled  version  is  available  for  the  WBT.  The  purpose  of  this  assessment  is  twofold.  First, 
it  provides  feedback  to  students  on  their  perceptions  of  their  social  awareness  and  influence 
skills.  This  feedback  helps  to  raise  self-awareness  and  further  engage  students  in  the  course 
contents.  The  second  purpose  is  research.  The  assessment  enables  comparisons  between  self- 
assessments  and  more  objective  assessments  of  social  awareness  and  influence  skills. 

The  assessment  items  were  developed  by  the  project  team  to  reflect  each  aspect  of  the  social 
awareness  and  influence  measurement  model.  All  members  of  the  team  had  extensive  experience 
researching  the  topics  of  social  awareness  and  influence  and  used  relevant  literature  to  generate 
the  items.  Respondents  answer  each  question  on  a  five  point  Likert  scale  ranging  from  Strongly 
Disagree  to  Strongly  Agree. 

Knowledge  Test.  The  Knowledge  Test  is  a  20-item  multiple  choice  exam  that  is  based  on 
the  contents  of  the  course.  A  paper  version  of  this  assessment  is  available  for  the  classroom 
course  and  a  web-enabled  version  is  available  for  the  WBT.  The  purpose  of  the  knowledge  test  is 
to  assess  whether  students  have  learned  important  concepts  presented  in  the  course.  The 
Knowledge  Test  was  written  by  the  course  developers  and  includes  items  that  address  each  of  the 
course  modules.  This  test  was  developed  after  the  classroom  course  was  piloted,  so  data  are  only 
available  for  the  web-based  sample. 

SJT.  The  situational  judgment  test  (SJT)  includes  a  video-based  scenario  with  20  multiple 
choice  questions.  A  paper  version  of  the  questions  is  available  for  the  classroom  course,  and  a 
web-enabled  version  is  available  for  the  WBT.  The  video  scenario  begins  with  a  brief  overview 
of  the  influence  scenario,  followed  by  several  multiple  choice  questions.  Next,  four  video 
segments  of  dialog  between  two  characters  (an  influencer  and  a  target)  are  presented;  each 
followed  by  several  multiple  choice  questions.  The  purpose  of  the  SJT  is  to  assess  the  extent  to 
which  students  demonstrate  social  insight  in  a  complex  interpersonal  interaction.  Specifically, 
the  SJT  was  designed  to  assess  the  students’  ability  to  effectively  plan  an  influence  attempt, 
interpret  the  reactions  of  a  target,  evaluate  these  reactions  against  the  influence  goal,  and  make 
decisions  about  how  to  interact  with  the  target  to  accomplish  the  influence  goal.  The  SJT  was 
developed  based  on  the  Social  Awareness  and  Influence  Measurement  Model  presented  in  Table 
1. 


The  scenario  chosen  for  the  SJT  was  a  downward  influence  attempt.  In  the  scenario,  a  new 
lieutenant  is  trying  to  get  buy-in  and  support  from  one  of  his  NCOs  in  handling  a  performance 
problem  with  one  of  the  corporals  in  the  unit.  Although  the  lieutenant  recognizes  that  he  can  use 
his  authority  to  deal  with  the  performance  issue,  he  recognizes  that  getting  the  NCO’s  support  for 
his  decision  is  important  to  preserving  the  morale  of  the  unit  and  ensuring  his  plan  is  actually 
carried  out  in  an  effective  manner.  The  scenario  and  responses  were  developed  based  on  input 
from  Solider  SMEs  who  participated  in  the  focus  groups  described  above.  This  scenario  was 
chosen  because  the  Soldier  SMEs  reported  that  downward  influence  is  a  challenging  problem 
faced  by  junior  leaders. 
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The  SJT  script  was  filmed  using  digital  photography  to  capture  still  photos  that  represented 
the  facial  expressions  of  the  two  characters.  Stock  photos  of  Army  personnel  in  Iraq  were  used 
for  the  introduction.  Voice-over  narration  was  used  to  capture  the  introduction  text  and  the 
dialog  between  the  two  characters.  Each  question  in  the  SJT  links  back  to  a  particular  step  in  the 
measurement  model,  and  the  response  options  are  scaled  (using  expert  judgments)  to  indicate  the 
effectiveness  of  each  option. 

Eight  Industrial/Organizational  psychologists  with  extensive  experience  in  social  awareness 
and  influence  research  served  as  the  SMEs  for  scaling  the  response  options.  The  SMEs  rated 
each  response  in  the  level  of  insight  and  judgment  that  the  response  indicated.  Specifically,  raters 
reviewed  each  item  on  the  SJT  and  were  instructed  to,  “ Rate  each  option  (in  terms  of  how 
accurate  or  effective  that  option  is)  for  each  question,  using  the  following  scale:  1  =  poor  social 
insight  and  judgment,  2  =  a  minimal  amount  of  social  insight  and  judgment,  3  =  a  moderate 
degree  of  social  insight  and  judgment,  4  =  a  very  high  degree  of  social  insight  and  judgment .” 
The  mean  of  these  eight  ratings  for  each  response  option  was  used  to  develop  the  scoring  system 
for  the  SJT.  For  example,  if  a  participant  selected  response  option  a  and  that  response  option  had 
a  mean  rating  of  3.80;  the  participant  would  receive  3.80  points  for  that  question.  The  points 
would  then  be  summed  to  compute  a  final  score  on  the  SJT. 

This  scoring  scheme  was  applied  to  determine  scores  for  the  participants  in  a  classroom  pilot 
test  conducted  in  June  of  2007  (see  the  next  section  for  details).  Following  this  pilot  test,  more 
detailed  analyses  were  conducted  to  refine  the  items  and  scoring  for  the  web-based  version  of  the 
SJT.  To  refine  the  scoring  for  the  web-based  version  of  the  SJT,  first  the  highest  and  lowest  SME 
ratings  for  each  option  were  dropped  from  analysis,  leaving  six  ratings  for  each  option  (Weekly, 
Ployhart,  &  Holtz,  2006).  This  was  done  to  eliminate  outliers.  Next,  we  tested  rater  agreement. 

To  test  the  agreement  of  the  SME  ratings,  the  Average  Deviation  Index  (ADI)  (Burke, 
Finkelstein,  &  Dusig,  1999)  was  computed  for  each  response  option.  The  ADI  is  the  mean  of  the 
absolute  deviations  of  the  responses  from  the  mean.1  This  index  measures  the  extent  to  which 
respondents  agree  in  their  ratings  of  the  response  options,  and  it  was  used  to  adjust  the  items  that 
had  response  ratings  with  low  agreement.  Response  options  that  had  an  ADI  greater  than  .67 
were  considered  problematic  based  on  Burke  and  Dunlap’s  (2002)  recommendations.  Burke  and 
Dunlap  demonstrated  that  dividing  the  number  of  response  categories  by  six  would  result  in  a 
value  that  corresponded  roughly  to  a  reliability  of  .70.  In  this  case  this  would  result  in  a  cutoff  of 
.67,  given  that  there  were  four  potential  ratings  for  each  option.  We  removed  responses  with  ADI 
values  above  .67  from  their  respective  items. 

These  criteria  resulted  in  several  adjustments  to  the  SJT  items.  Option  E  was  removed  from 
question  3,  option  D  was  removed  from  question  10,  option  C  was  removed  from  question  14, 
option  A  was  removed  from  question  16,  and  option  D  was  removed  from  question  18.  Since 
there  was  poor  agreement  on  4  of  the  5  response  options  for  question  7,  question  7  was  removed 


1  Note:  see  Burke  and  Dunlap  (2002)  for  a  thorough  explanation  of  the  ADI  and  the  merits  of  this  index  in 
comparison  with  other  methods  of  measuring  interrater  agreement. 
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from  the  scoring  of  the  SJT  but  was  kept  in  the  test  for  continuity.2  All  of  the  remaining  items 
and  response  options  showed  adequate  agreement  among  expert  raters  and  were  therefore 
included  in  the  final  version  of  the  test.  Point  values  for  each  option  were  calculated  using  the 
mean  of  the  six  SME  ratings  after  the  adjustments  described  above.  The  results  from  these 
analyses  and  the  final  point  values  used  to  calculate  SJT  assessment  scores  are  presented  in 
Table  4. 

Table  4.  Agreement  Indices  and  Final  Scores  for  SJT  test  items 


Item 

Response 

Option 

ADI 

Response  Option 
Point  Value 

Item 

Response 

Option 

ADI 

Response  Option 
Point  Value 

1A 

.28 

3.83 

11A 

.56 

1.83 

IB 

.28 

2.17 

11B 

.28 

3.83 

1C 

.44 

2.67 

11C 

.44 

1.67 

ID 

.67 

1.50 

11D 

.44 

1.67 

IE 

.44 

1.67 

HE 

.00 

1.00 

2A 

.50 

2.50 

12A 

.00 

4.00 

2B 

.50 

1.50 

12B 

.28 

2.17 

2C 

.44 

2.67 

12C 

.28 

1.17 

2D 

.00 

4.00 

12D 

.44 

1.67 

2E 

.28 

1.83 

12E 

.44 

1.33 

3A 

.00 

1.00 

13A 

.00 

4.00 

3B 

.44 

1.33 

13B 

.44 

2.67 

3C 

.28 

3.83 

13C 

.28 

3.17 

3D 

.33 

2.00 

13D 

.44 

1.33 

3E 

.83 

0.00 

13E 

.00 

1.00 

4A 

.44 

2.67 

14A 

.28 

1.17 

4B 

.00 

4.00 

14B 

.50 

2.50 

4C 

.28 

1.83 

14C 

.83 

0.00 

4D 

.00 

1.00 

14D 

.28 

3.83 

4E 

.33 

2.00 

14E 

.44 

3.33 

5A 

.44 

2.33 

15A 

.67 

1.67 

5B 

.50 

2.50 

15B 

.67 

3.50 

5C 

.00 

4.00 

15C 

.33 

2.00 

5D 

.00 

1.00 

15D 

.44 

2.67 

5E 

.67 

1.67 

15E 

.28 

1.17 

6A 

.00 

1.00 

16A 

.83 

0.00 

6B 

.67 

2.67 

16B 

.44 

1.67 

6C 

.44 

1.67 

16C 

.00 

4.00 

6D 

.56 

1.83 

16D 

.00 

1.00 

6E 

.28 

3.83 

16E 

.50 

1.50 

2  Note:  question  7  asked  respondents  to  indicate  what  one  of  the  characters  in  the  scenario  would  likely  do  next,  and 
it  is  probably  that  not  enough  information  about  the  characters  had  been  provided  yet  at  that  point  in  the  video. 
Subsequent  repetitions  of  the  same  questions  at  later  points  in  the  video  yielded  better  results. 
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Item 

Response 

Option 

ADI 

Response  Option 
Point  Value 

Item 

Response 

Option 

ADI 

Response  Option 
Point  Value 

7A 

.89 

0.00 

17A 

.67 

1.50 

7B 

.78 

0.00 

17B 

.56 

3.17 

7C 

.89 

0.00 

17C 

.67 

3.33 

7D 

.83 

0.00 

17D 

.67 

3.33 

7E 

.50 

0.00 

17E 

.44 

2.67 

8A 

.28 

1.17 

18A 

.28 

2.83 

8B 

.33 

2.00 

18B 

.00 

4.00 

8C 

.44 

2.33 

18C 

.00 

1.00 

8D 

.28 

1.17 

18D 

.78 

0.00 

8E 

.00 

4.00 

18E 

.00 

1.00 

9A 

.67 

3.33 

19A 

.00 

4.00 

9B 

.28 

1.17 

19B 

.44 

2.33 

9C 

.44 

3.67 

19C 

.00 

1.00 

9D 

.28 

1.17 

19D 

.44 

1.33 

9E 

.00 

1.00 

19E 

.00 

1.00 

10A 

.00 

1.00 

20A 

.00 

1.00 

10B 

.44 

2.67 

20B 

.00 

1.00 

10C 

.56 

2.17 

20C 

.33 

2.00 

10D 

.78 

0.00 

20D 

.50 

3.50 

10E 

.28 

3.83 

20E 

.44 

3.67 

Note:  Items  with  high  ADI  values  and  low  rwg  values  are  shaded. 


Web-based  Role-Play.  The  web-based  role-play  represents  the  automation  of  one  of  the 
live-action  role-plays  used  in  the  classroom  course.  The  purpose  of  the  web-based  role-play  was 
to  assess  the  extent  to  which  students  demonstrate  social  insight  in  a  complex  interpersonal 
interaction.  Specifically,  the  web-based  role-play  was  designed  to  assess  the  students’  ability  to 
effectively  plan  an  influence  attempt,  interpret  the  reactions  of  a  target,  evaluate  these  reactions 
against  the  influence  goal,  and  make  decisions  about  how  to  interact  with  the  target  to 
accomplish  the  influence  goal.  It  includes  a  video-based  scenario  with  several  multiple  choice 
questions.  The  role-play  begins  with  a  brief  video  providing  an  overview  of  the  influence 
scenario  followed  by  several  multiple  choice  questions.  Next,  the  student  engages  in  a  dialog 
with  the  central  character,  an  Iraqi  Mukhtar.  After  each  statement  by  the  Mukhtar,  the  student  is 
offered  2-4  response  options  from  which  to  choose.  The  scenario  branches,  such  that  the  students 
are  presented  with  different  dialog  from  the  Mukhtar  and  different  subsequent  response  options 
depending  on  the  response  option  selected.  The  web-based  role-play  was  developed  based  on  the 
Social  Awareness  and  Influence  Measurement  Model  presented  in  Table  1. 

The  scenario  selected  for  the  web-based  role-play  was  an  influence  attempt  based  upon 
examples  obtained  in  the  focus  groups  with  subject  matter  experts  described  earlier;  this  helped 
to  make  the  role-play  as  realistic  as  possible.  Role-plays  have  been  identified  as  a  meaningful 
way  of  teaching  interpersonal  skills  (Ahamer,  2004;  Shortridge  &  Sabo,  2005;  Nelson  & 

Blenkin,  2007)  as  well  as  measuring  the  success  of  influence  attempts  (Yukl,  Kim,  &  Chavez, 
1999).  Web-based  role-plays  have  also  been  argued  to  be  beneficial  in  developing  interpersonal 
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skills  (Olson-Buchanan,  Drasgow,  Moberg,  Mead,  Keenan,  &  Donovan,  1998).  Web-based  role- 
plays  may  be  particularly  useful  in  the  assessment  and  development  of  interpersonal  skills 
because  they  do  not  have  the  problems  of  low  interrater  reliability  associated  with  more 
traditional  role-plays  because  they  can  be  presented  in  a  standardized  fonnat  that  has  more 
realism  than  paper  and  pencil  assessments  (Olson-Buchanan  et  al.,  1998). 

Similar  to  the  SJT,  the  video  for  the  role-play  was  filmed  using  digital  photography  to 
capture  still  photos  of  an  actor  playing  the  part  of  the  Mukhtar  in  order  to  convey  emotional  cues 
through  facial  expressions  and  body  language.  Voice  over  narration  was  used  to  capture  each 
response  by  the  Mukhtar  as  well  as  to  provide  a  detailed  explanation  of  the  scenario  before  the 
role-play  began.  After  presenting  an  introduction  to  the  scenario  that  was  accompanied  by 
pictures  of  the  geographical  area  in  which  the  scene  took  place,  participants  were  asked  a  few 
questions  to  measure  the  degree  to  which  they  understood  the  situation  from  the  description 
provided.  These  included  questions  related  to  the  identification  of  their  primary  goals  as  well  as 
the  factors  that  would  be  most  important  for  them  to  consider  during  the  role-play. 

Similar  to  other  web-based  role-plays  (Olson-Buchanan,  2002;  Olson-Buchanan  et  al.,  1998) 
branching  was  used  extensively.  In  a  branching  role-play,  the  assessment  branches  to  images  and 
audio  that  provide  feedback  to  the  participant  based  upon  the  action  that  he/she  has  selected 
(Olson-Buchanan,  2002).  At  critical  points  in  the  role-play  (following  responses  from  the 
Mukhtar)  the  role-play  would  be  stopped  and  participants  would  be  provided  with  written 
descriptions  of  four  different  response  options.  Based  upon  the  Social  Awareness  and  Influence 
Measurement  Model  each  response  option  reflected  varying  degrees  of  two  factors:  (1)  a 
reflection  of  the  degree  of  awareness  of  the  emotions  expressed  by  the  Mukhtar,  and  (2)  the  type 
of  influence  tactic  selected  for  achieving  their  objective  (tactics  ranged  in  appropriateness  for  the 
situation  as  based  upon  the  Social  Awareness  Model).  Typically,  three  to  four  types  of  responses 
were  possible:  high  social  awareness  and  a  highly  appropriate  influence  tactic,  high  social 
awareness  and  a  less  appropriate  tactic,  low  social  awareness  and  a  highly  appropriate  tactic 
(optional),  and  low  social  awareness  and  an  inappropriate  tactic. 

The  role-play  was  designed  in  such  a  manner  that  the  response  selected  by  the  participant 
would  determine  the  feedback  that  he/she  received  from  the  Mukhtar  as  well  as  future  response 
options  that  would  be  available.  Thus,  the  scenario  changed  based  upon  the  responses  selected  by 
the  participant;  each  Mukhtar  clip  was  a  direct  result  of  the  response  option  selected  by  the 
participant.  Based  upon  whether  an  individual  selected  a  “correct”  or  “incorrect”  response  they 
would  receive  a  different  follow-up  screen.  However,  regardless  of  what  response  options  were 
selected  each  participant  was  asked  to  select  a  response  at  18  different  points  in  the  role-play. 
While  it  was  likely  that  participants  received  distinctly  different  follow-up  scenes  with  their 
accompanying  response  choices,  it  was  also  possible  to  recover  from  an  initial  “mistake”  by 
making  subsequent  “correct”  decisions  that  would  lead  the  participant  to  the  same  scene  reached 
by  an  individual  making  more  “correct”  decisions.  Similarly,  a  participant  could  initially  make  a 
“correct”  decision  and  still  receive  the  same  scene  as  someone  consistently  making  “poor” 
choices  if  they  subsequently  selected  “incorrect”  response  options.  This  was  important  in  order 
to  make  the  role-play  as  realistic  as  possible;  in  actual  influence  attempts  it  is  possible  to  achieve 
one’s  objectives  even  if  initial  attempts  fail  (and  vice  versa). 
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The  scoring  of  web-based  role-plays  using  branching  can  be  particularly  problematic  (Olson- 
Buchanan,  2002;  Drasgow,  Olson-Buchanan,  &  Moberg,  1999;  Olson-Buchanan  et  ah,  1998). 
Specifically,  a  decision  has  to  be  made  on  how  to  assign  points  to  the  response  option  selected  in 
a  follow-up  scene  that  is  presented  based  upon  an  earlier  “correct”  response,  versus  one  that 
would  be  presented  based  upon  an  earlier  “incorrect”  response  (Olson-Buchanan,  2002).  This 
issue  was  addressed  in  a  similar  manner  to  other  computer  based  role-plays  by  using  a  clean  slate 
approach  to  determine  scoring  (Drasgow  et  ah,  1999;  Olson-Buchanan  et  ah,  1998).  In  a  clean 
slate  approach,  each  response  option  selected  by  a  participant  is  scored  independently  of  any  of 
his  or  her  other  selections.  In  this  case,  each  response  received  a  score  from  zero  to  two;  a  zero 
was  assigned  to  responses  reflecting  neither  social  awareness  nor  the  selection  of  an  appropriate 
tactic,  a  one  was  assigned  to  those  options  reflecting  either  social  awareness  or  the  selection  of 
an  appropriate  tactic,  and  a  two  was  assigned  to  options  reflecting  both  social  awareness  and  the 
selection  of  an  appropriate  tactic.  The  scoring  of  all  response  options  was  determined  by 
agreement  between  three  industrial/organizational  psychologists  familiar  with  the  model  of 
Social  Awareness  who  were  familiar  with  the  dimensions  (emotional  awareness,  influence  tactic 
appropriateness)  that  the  scoring  was  based  upon.  Any  disagreements  were  addressed  through 
the  rewording  of  response  options  as  necessary. 

Finally,  a  regional  expert,  Dr.  Alma  Abdul-Hadi  Jadallah,  reviewed  the  role-play  in  its 
entirety  before  production  began  and  provided  extensive  guidance  on  the  script  and  response 
options.  Olson-Buchanan  et  al.  (1998)  argued  that  in  order  to  use  model-based  scoring  as  we 
have  done  that  it  is  necessary  for  each  branch  to  be  realistic  and  congruent  with  the  model  that 
will  be  used  for  scoring.  This  was  accomplished  through  the  cultural  review  and  the  review  of  all 
response  options  by  the  Indus  trial/ Organizational  psychologists  familiar  with  the  social 
awareness  model.  As  a  result,  a  model-based  scoring  scheme  was  used. 

Reliability  was  not  calculated  for  this  assessment  of  the  WBT,  which  is  not  uncommon 
for  role-plays  that  employ  the  use  of  branching  (Olson-Buchanan,  2002;  Drasgow  et  al.,  1999; 
Olson-Buchanan  et  al.,  1998).  Namely,  each  scene  of  the  role-play  with  the  associated  response 
options  is  not  viewed  by  every  participant.  As  a  result,  because  participants  do  not  each  receive 
the  same  exact  items,  it  is  inappropriate  to  calculate  a  coefficient  alpha  value  for  this  assessment 
(Olson-Buchanan,  2002).  Similarly,  it  is  not  feasible  to  calculate  test-retest  reliability  because 
participants  are  likely  to  select  different  response  options  from  one  testing  session  to  another, 
and  thus  be  presented  with  a  different  set  of  items.  At  present  there  does  not  appear  to  be  an 
agreed  upon  method  that  is  feasible  for  assessing  reliability  on  adaptive  assessments;  this  is  an 
area  that  has  been  identified  as  being  important  for  future  research  as  a  result  (Olson-Buchanan, 
2002).  While  the  use  of  IRT  could  be  useful  in  the  design  of  a  branching  role-play,  the  decision 
to  use  model  based  scoring  and  the  fact  that  there  was  not  a  sufficient  sample  size  of  role-play 
scenes  or  participants  made  this  infeasible  for  this  project.  As  a  result,  no  reliability  estimates  are 
reported  for  the  web-based  role-play. 

Pilot  Testing 

Classroom  POI.  The  final  revised  classroom  POI  was  pilot  tested  at  location  1  in  June  of 
2007  with  a  total  of  48  enlisted  Soldiers  and  officers.  Demographic  information  for  this  sample  is 
provided  in  Tables  5,  6,  and  7.  The  pilot  test  included  the  following  components: 
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1.  The  purpose  of  the  pilot  test  was  explained  and  Soldiers  signed  privacy  act  statements. 

2.  Soldiers  completed  the  Skills  Self-Assessment. 

3.  Soldiers  completed  the  video-based  SJT.  In  half  of  the  classes,  the  students  were  randomly 
assigned  to  take  the  criterion  measure  (Corporal  Bennett  SJT),  which  served  as  a  pre-test. 

The  other  half  were  assigned  to  take  another  video-based  SJT  that  served  as  a  filler  activity. 

4.  Course  contents  were  presented  as  described  in  the  outline  in  Table  1. 

5.  Soldiers  completed  the  SJT  post-test.  The  Soldiers  who  had  completed  the  filler  activity  in 
the  morning  completed  the  Corporal  Bennett  SJT  and  the  Soldiers  who  had  completed  the 
Corporal  Bennett  SJT  in  the  morning  completed  the  filler  activity. 

6.  Soldiers  completed  the  post-course  reactions  questionnaire  and  were  released. 

WBT  POI.  The  final  web-based  POI  was  pilot  tested  at  location  2  in  May  of  2008  with  a 
total  of  41  officers.  Demographic  infonnation  for  this  sample  is  provided  in  Tables  5,  6,  and  7. 
Each  team  of  students  was  randomly  assigned  to  the  training  condition  or  the  control  condition. 
In  the  training  condition,  participants  completed  the  web-based  course  (which  included  the 
Social  Awareness  and  Influence  Skills  Self-assessment)  and  then  the  post-course  reactions 
questionnaire.  Then  participants  completed  three  criterion  measures:  the  knowledge  test,  the  SJT, 
and  the  online  role-play.  In  the  control  condition,  participants  completed  the  criterion  measures 
first,  followed  by  the  training  course.  In  one  of  the  control  sessions,  access  to  the  web-site 
containing  the  training  contents  was  temporarily  disrupted  due  to  technical  difficulties. 

Therefore,  9  participants  were  unable  to  complete  the  training.  Participants  completed  the 
training  and  assessments  at  their  own  pace.  The  time  to  complete  the  training  and  assessments 
ranged  from  4-5  hours. 

Table  5.  Rank  of  Samples 


WBT 

(Location  2) 

Classroom  Training 
(Location  1) 

Control 

Group 

Training 

Group 

Control 

Group 

Training 

Group 

Rank 

Freq 

Valid 

% 

Freq 

Valid 

% 

Freq 

Valid 

% 

Freq 

Valid 

% 

Private  First  Class 

0 

0 

0 

0 

0 

0 

1 

3.9% 

Specialist 

0 

0 

0 

0 

0 

0 

9 

34.6% 

Sergeant 

0 

0 

0 

0 

1 

4.6% 

5 

19.2% 

Staff  Sergeant 

0 

0 

0 

0 

9 

40.9% 

1 

3.9% 

Sergeant  First 

Class 

0 

0 

0 

0 

4 

18.2% 

0 

0 

Second  Lieutenant 

1 

5% 

0 

0 

2 

9.1% 

0 

0 

First  Lieutenant 

10 

50% 

7 

33.3% 

3 

13.6% 

0 

0 

Captain 

6 

30% 

11 

52.4% 

3 

13.6% 

10 

38.5% 

Major 

3 

15% 

3 

14.3% 

0 

0 

0 

0 
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Results 


Descriptive  statistics  for  the  classroom  group  variables  are  presented  in  Table  8.  Correlations 
among  relevant  variables  in  the  classroom  group  are  presented  in  Table  9.  Descriptive  statistics 
for  the  classroom  group  variables  are  presented  in  Table  10.  Correlations  among  relevant 
variables  in  the  classroom  group  are  presented  in  Table  11.  Significant  findings  are  discussed  in 
more  detail  in  the  sections  that  follow. 

As  noted  previously,  the  classroom  based  course  and  web-based  course  were  developed  a 
year  apart.  Therefore,  while  the  self-assessment  and  SJT  content  and  questions  were  the  same  in 
both  versions,  the  knowledge  test  and  scored  role-play  were  only  available  for  the  web-based 
sample. 

Table  6.  Descriptive  Statistics  of  Classroom  Sample  Variables 


N 

Min 

Max 

M 

SD 

Alpha 

Social  Awareness  and 
Influence  Self-Assessment 

48 

20 

48 

36.60 

6.01 

.78 

(Influence  Scale) 

Social  Awareness  and 
Influence  Self-Assessment 

48 

23 

46 

35.56 

4.41 

.72 

(Social  Awareness  Scale) 
Social  Awareness  and 
Influence  Self-Assessment 

48 

50 

93 

72.17 

9.39 

.85 

(Total  Score) 

Total  SJT  Score 

43 

48.18 

72.93 

64.75 

5.07 

n/a 

Table  7.  Correlations  Among  Classroom  Sample  Variables 


Influence 

Scale 

Social 

Self- 

Rank 

Awareness 

Scale 

Assessment 

Total 

Social  Awareness  and 
Influence  Self-Assessment 
(Influence  Scale) 

Social  Awareness  and 

0.19 

Influence  Self-Assessment 
(Social  Awareness  Scale) 
Social  Awareness  and 

0.37 

0.62 

Influence  Self-Assessment 

0.29 

0.93 

0.86 

(Total  Score) 

Total  SJT  Score 

0.35 

0.36 

0.47 

0.45 

Note:  scores  in  bold  are  statistically  significant,/?  <  .05.;  N=  48  for  Self-Assessments  and  43  for 


SJT 
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Table  8.  Descriptive  Statistics  of  Web-based  Sample  Variables 


N 

Min 

Max 

M 

SD 

Alpha 

Social  Awareness  and 

Influence  Self-Assessment 

41 

23 

50 

37.68 

5.58 

.85 

(Influence  Scale) 

Social  Awareness  and 

Influence  Self-Assessment 

41 

26 

48 

37.80 

4.69 

.78 

(Social  Awareness  Scale) 
Social  Awareness  and 

41 

49 

98 

75.49 

9.98 

.91 

Influence  Self-Assessment 
(Total  Score) 

Total  SJT  Score 

41 

49.67 

71.98 

64.63 

4.17 

n/a 

Total  Knowledge  Test  Score 

41 

9 

20 

14.17 

3.29 

.70 

Total  Role-Play  Score 

33 

33 

43 

38.21 

2.77 

n/a 

Table  9.  Correlations  Among  Online  Sample  Variables 


Influence 

Scale 

Social 

Self- 

Total 

Total 

Rank 

Awareness 

Assessment 

SJT 

Knowledge 

Scale 

Total 

Score 

Test  Score 

Social  Awareness  and 
Influence  Self- 
Assessment  (Influence 
Scale) 

Social  Awareness  and 
Influence  Self- 
Assessment  (Social 
Awareness  Scale) 

Social  Awareness  and 
Influence  Self- 
Assessment 

.04 

.02 

.89 

.97 

.03 

.98 

(Total  Score) 

Total  SJT  Score 

.03 

.06 

.17 

.12 

Total  Knowledge  Test 
Score 

.11 

.13 

.11 

.13 

.11 

Total  Role-Play  Score 

-.44 

.04 

.13 

.08 

.37 

-.07 

Note:  scores  in  bold  are  statistically  significant,/;  <  .05.  N=  41  for  the  Self-Assessments  and 
knowledge  test  and  33  for  the  role-play. 


Participant  Satisfaction 

Summaries  of  participant  responses  on  the  course  evaluation  questionnaires  for  both  the 
classroom  course  and  the  WBT  prototype  are  provided  in  Tables  12,  13,  14,  and  15.  Responses 
to  general  satisfaction  questions  in  Table  12  were  on  a  five-point  Likert  Scale  (5  =  Strongly 
Agree;  1  =  Strongly  Disagree).  Responses  in  Table  13  were  regarding  the  difficulty  of  the  course 
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contents,  and  responses  in  Table  14  were  regarding  the  length  of  the  course.  A  summary  of  open- 
ended  comments  is  presented  in  Table  15.  Complete  results  of  satisfaction  questionnaires  are 
provided  in  Appendix  A  (classroom)  and  Appendix  B  (WBT). 


Table  10:  Satisfaction  Ratings 


Classroom 

WBT 

M 

SD 

M 

SD 

This  course  helped  me  to  better  understand  the  importance 
of  social  awareness  and  influence  to  Army  leaders. 

4.10 

.692 

4.09 

.689 

This  course  helped  me  to  learn  more  about  my  own 
strengths  and  weaknesses  related  to  social  awareness  and 
influence. 

4.00 

.684 

3.91 

.818 

This  course  helped  me  to  better  understand  the  social 
awareness  and  influence  process. 

4.15 

.618 

4.28 

.683 

This  course  helped  me  to  better  recognize  and  accurately 
interpret  verbal  and  nonverbal  cues  from  others. 

3.96 

.713 

3.84 

.860 

This  course  taught  me  how  to  change  my  own  behavior  to 
more  effectively  interact  with  other  people. 

3.88 

.733 

3.72 

.851 

This  course  helped  me  to  become  a  better  influencer. 

3.92 

.739 

3.78 

.906 

Overall,  I  believe  that  what  I  learned  in  this  course  will 
help  me  to  be  a  better  leader. 

4.06 

.727 

4.13 

.707 

Overall,  I  believe  this  course  was  worthwhile. 

4.10 

.831 

4.12 

.751 

I’m  planning  to  use  what  I  learned  in  this  course  in  future 
interactions  with  other  people  on  the  job. 

4.19 

.680 

4.19 

.821 

Table  11.  Difficulty  of  Course  Contents 

Classroom 

WBT 

„  Valid 

Valid 

Frequency  „ 

Percent 

Frequency  _ 

Percent 

Too  complex  or  difficult  2  4.3% 

0 

0 

Too  basic  or  simple  4  8.5% 

4 

12.5% 

Appropriate  -  neither  too  basic  nor  too  . ,  _0/ 

,  41  O/.2/0 

28 

87.5% 

compiex 
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Table  12.  Length  of  Course 


Classroom 

WBT 

Frequency 

Valid 

Percent 

Frequency 

Valid 

Percent 

Too  long  for  the  amount  of  material 
presented 

4 

8.5% 

4 

12.5% 

Too  short  for  the  amount  of  material 
presented 

1 

2.1% 

3 

9.4% 

Appropriate  -  neither  too  long  nor  too 
short 

42 

89.4% 

25 

78.1% 

Table  13.  Summary  of  Open-Ended  Comments 


What  was  the  most  useful  part  of  the  course 
and  why? 

What  was  the  least  useful  part  of  the  course 
and  why? 

Classroom  1 

♦  Role-plays  and  scenarios 

♦  Tips  on  influence  tactics 

♦  Discussion 

♦  Learning  more  about  social  awareness  and 
influence 

♦  Videos 

♦  Power-Point  Slides 

♦  Facial  expressions 

WBT  | 

♦  Realistic  military  examples  and  vignettes 

♦  Scenarios 

♦  Learning  checks 

♦  Useful  content 

♦  Taking  the  course  on-line  instead  of 
classroom  (e.g.,  sometimes  made  it  hard  to 
stay  engaged;  missed  discussion) 

♦  Lack  of  audio  in  the  course 

♦  Technical  glitches  (e.g.,  slow  downloads; 
program  freezing) 

Criterion  Measures 

Knowledge  Test.  In  the  WBT  sample,  the  mean  score  on  the  knowledge  test  was  16.19 
(, sd  =  2.94)  for  the  training  group  and  12.05  for  the  control  group  (sd  =  2.09).  This  difference  was 
significant  (t  =  5.17,  df=  39,  /;<.()() 1 ).  Thus  the  manipulation  check  was  successful;  those  in  the 
training  group  acquired  significantly  more  knowledge  related  to  the  course  content  than  those  in 
the  control  condition. 

SJT.  The  mean  score  on  the  SJT  for  classroom  training  was  65.06  (sd  =  5.22)  for  the 
group  that  took  the  SJT  as  a  pre-test  measure  and  64.47  (sd  =  5.04)  for  the  group  that  took  the 
SJT  as  a  post-test  measure.  This  difference  was  not  statistically  significant.  On  closer 
examination,  it  appeared  that  the  SJT  results  were  correlated  with  rank,  and  the  ranks  were  not 
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evenly  distributed  between  these  two  groups.  Therefore,  these  two  groups  differed  in  an 
important  way  and  were  not  equivalent,  making  pre-post  comparisons  problematic.  A  follow-up 
ANCOVA  to  control  for  the  effects  of  rank  showed  that  rank  rather  than  group  membership 
accounts  for  nearly  all  the  variance  in  performance  [/( 1 ,45)  =  6.24 1 ,  /;=.()  1 6  for  rank  and  /(  1 . 45) 
=  .014,  ns  for  group].  However,  the  small  sample  size  and  wide  dispersion  of  ranks  in  the 
classroom  sample  makes  this  finding  difficult  to  interpret. 

In  the  WBT  sample,  the  mean  score  on  the  SJT  was  63.85  ( sd  =3.78)  in  the  training  group 
and  65.49  ( sd  =  4.49)  in  the  control  group.  These  two  scores  are  not  significantly  different.  In 
this  sample,  the  SJT  scores  were  not  correlated  with  self-reported  skills,  knowledge  test  results, 
or  rank.  However,  SJT  scores  were  correlated  with  role-play  performance  (see  Table  10). 

Web-based  Role-Play.  Results  for  the  web-based  role-play  scores  were  somewhat 
problematic  because  20%  of  respondents  were  missing  data  (final  n  =  33);  it  is  not  clear  whether 
this  data  was  missing  due  to  an  electronic  error  in  the  data  collection  process  or  whether  these 
participants  simply  did  not  complete  the  role-play.  Therefore,  analyses  were  only  conducted  on 
the  respondents  who  had  complete  data.  No  significant  differences  were  observed  between 
scores  in  the  control  group  (m  =  38.89,  sd  =  2.17)  and  the  training  group  (m  =37.4,  sd  =  3.25). 
However,  as  shown  in  Table  10,  there  was  a  significant  difference  in  role-play  perfonnance  by 
rank  -  in  the  opposite  direction  that  might  have  been  predicted  based  on  the  results  of  the 
classroom  group.  That  is  rank  and  role-play  perfonnances  were  negatively  correlated  such  that 
lower  ranking  individuals  tended  to  score  higher  on  the  role-play  than  higher  ranking  individuals. 
A  follow-up  ANCOVA  to  control  for  the  effects  of  rank  showed  that  rank  rather  than  group 
membership  accounts  for  nearly  all  the  variance  in  performance  [/( 1 ,30)  =  5.749,  p=.023  for  rank 
and/(l,  30)  =  1.21,  ns  for  group]. 


Discussion 

Given  the  importance  of  social  awareness  and  influence  skills  to  Army  leaders,  ARI 
sponsored  a  multi-year  effort  to  develop  a  model  of  the  social  awareness  and  influence  process 
and  associated  training  programs  based  upon  principles  from  this  model  to  help  Anny  leaders 
enhance  their  skills  in  these  areas.  As  part  of  this  effort,  a  classroom-based  course  and  a 
prototype  web-based  course  were  developed  along  with  assessments  of  social  awareness  and 
influence  skills.  These  programs  were  then  pilot  tested  with  field  samples  of  Army  officers  and 
NCOs. 

The  pilot  studies  served  three  major  purposes:  (1)  to  test  the  various  training  concepts  to 
determine  their  suitability  for  ongoing  use,  (2)  to  determine  if  social  awareness  and  influence 
skills  could  be  reliably  measured,  and  (3)  to  make  some  initial  determinations  about  whether 
social  awareness  and  influence  skills  could  be  improved  through  training.  Evaluation  data  were 
collected  at  two  levels  to  assess  the  effectiveness  of  the  training:  participant  reactions  (level  1) 
and  acquisition  of  new  knowledge  and  skills  (level  2).  The  level  1  evaluation  was  conducted 
using  a  survey  administered  at  the  end  of  each  training  session.  The  level  2  evaluation  consisted 
of  a  knowledge  test,  a  situational  judgment  test,  and  a  role-play  assessment. 
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With  regard  to  the  question  of  the  suitability  of  training  concepts,  results  of  the  participant 
reactions  questionnaire  indicated  that  the  training  programs  in  both  the  classroom  and  web-based 
formats  were  well-received  by  the  target  audience.  Participants  in  both  the  WBT  and  the 
classroom  training  reported  similar  satisfaction  ratings.  However,  while  individuals  may  have 
been  expected  to  prefer  WBT  (Hays,  2005),  50%  of  the  participants  who  completed  the  WBT 
indicated  a  preference  for  learning  this  content  via  classroom  training.  This  is  somewhat 
surprising  given  that  both  training  fonnats  were  rated  similarly  by  participants;  it  may  be  useful 
to  investigate  the  reasons  why  some  participants  reported  that  they  would  have  preferred  learning 
the  content  in  a  classroom  context.  Participants  in  both  samples  indicated  that  the  most  valuable 
aspects  of  the  course  included  the  use  of  real-world  examples  and  opportunities  to  practice  skills. 
It  appears  that  the  training  content  was  received  equally  well  for  the  most  part  across  both  the 
classroom  and  WBT  formats  and  participants  found  the  same  aspects  of  training  useful  in  both 
contexts. 

With  regard  to  the  measurement  of  social  awareness  skills,  results  indicate  promise  for  the 
measures  piloted  in  this  research.  The  self-report  measure  was  a  reliable  indicator  of  perceived 
social  awareness  and  influence  skills  (average  a  =  .78).  In  the  classroom  sample,  this  measure 
was  correlated  with  performance  on  the  judgment  test  (r  =.45,  p  <  .05).  In  the  web-based 
instruction  sample,  performance  on  the  situational  judgment  test  correlated  with  perfonnance  on 
the  automated  role-play  (r  =  .37,  p  <  .05).  As  these  measures  were  prototype  instruments,  more 
research  is  needed  to  validate  these  results  with  external  perfonnance.  However,  it  is  promising 
that  there  appears  to  be  some  initial  evidence  of  convergent  validity  for  these  skill  measures 

With  regard  to  the  trainability  of  social  awareness  and  influence  skills,  results  from  the  WBT 
sample  indicated  that  individuals  in  the  training  group  scored  significantly  higher  on  the 
knowledge  test  than  individuals  in  the  control  group.  Differences  in  situational  judgment  test 
scores  and  role-play  scores  were  not  significant  between  the  two  groups.  However,  in  Phase  I  of 
this  research  (Mueller-Hanson  et  ah,  2007)  significant  improvements  were  observed  in  person  to 
person  role-play  performance  before  and  after  the  training  when  the  training  was  presented  in  the 
classroom.  At  first  glance,  it  may  appear  puzzling  why  skills  were  improved  for  the  first 
classroom  sample  but  not  in  the  second  classroom  sample.  However,  the  role-play  assessment 
used  with  the  first  classroom  sample  is  very  different  from  the  video-based  SJT  used  in  the 
present  project.  It  may  be  that  success  in  the  SJT  is  more  related  to  cognitive  ability  than  success 
in  the  live  role-play,  which  could  in-part  account  for  the  differences  in  results. 

One  possible  conclusion  from  the  findings  in  the  current  project  is  that  a  short  online  training 
program  is  not  sufficient  for  building  new  skills,  but  it  is  useful  in  developing  knowledge. 
Therefore,  WBT  such  as  the  program  described  here  may  be  most  useful  as  pre-work  to  prepare 
students  for  more  intensive,  scenario-based  training  in  a  classroom  setting,  which  is  a  better 
forum  for  building  skills.  Another  possibility  is  that  the  person  to  person  and  web-based  role- 
play  assessments  differ  in  what  they  are  measuring.  It  is  also  possible  also  that  Soldiers  with 
different  levels  of  education  and  experience  may  respond  differently  to  different  training 
formats.  While  these  possibilities  future  consideration,  the  small  sample  sizes  and  wide 
dispersion  of  ranks  available  for  our  data  collection  makes  it  difficult  to  draw  firm  conclusions. 
Therefore,  it  would  be  fruitful  to  examine  the  relative  knowledge  and  skills  acquired  in 
classroom  and  WBT  training  within  a  given  rank  (or  narrow  band  of  ranks)  with  larger  samples 
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and  to  look  at  how  knowledge  and  skill  acquisition  may  differ  for  different  levels  of  education 
and  experience. 

With  regard  to  the  suitability  of  WBT  for  leadership  and  interpersonal  skills  development,  it 
may  be  unreasonable  to  expect  the  development  of  such  complex  skills  in  such  a  short  period  of 
time.  This  may  be  one  reason  why  even  though  participants  seemed  equally  pleased  with  the 
WBT  as  participants  with  the  classroom  training,  half  the  participants  who  completed  the  WBT 
indicated  a  preference  for  the  classroom  format.  These  participants  may  have  been  pleased  with 
the  training  they  received,  but  they  may  have  recognized  that  they  may  have  not  had  the 
opportunity  to  develop  these  skills  in  the  same  manner  they  might  have  in  the  classroom.  In  any 
event,  there  appears  to  be  value  in  both  training  formats.  The  WBT  can  more  easily  be 
administered  to  a  large  number  of  individuals  and,  as  discussed,  can  facilitate  the  development  of 
knowledge  related  to  influence  and  social  awareness.  It  may  be  best  to  follow  such  training  with 
a  more  traditional  classroom  based  training  that  can  focus  more  extensively  upon  the  enactment 
of  the  skills  necessary  for  influence  and  social  awareness. 

Conclusions  and  Limitations 

Several  conclusions  may  be  drawn  from  this  research.  First,  the  Social  Awareness  and 
Influence  Model  provides  a  useful  framework  for  research.  Several  hypotheses  may  be  drawn 
from  this  model  and  tested  empirically.  For  example,  one’s  ability  to  assess  a  situation  may  be 
related  to  one’s  ability  to  choose  an  appropriate  influence  goal  to  fit  that  situation.  Additionally, 
research  could  investigate  the  factors  of  the  situation  that  are  most  likely  to  be  recognized. 
Unfortunately,  the  small  sample  sizes  available  for  the  pilot  testing  prohibited  more  thorough 
testing  of  the  model.  It  would  be  useful  to  test  the  model  more  thoroughly  in  future  research. 

Second,  the  Social  Awareness  and  Influence  Model  provides  a  useful  framework  for  training 
social  awareness  and  influence  skills.  Students  reported  that  the  model  helped  them  to  think 
about  how  they  interact  with  others  in  a  more  structured  way.  As  one  individual  put  it,  “I  knew 
some  of  these  things  already,  but  you’ve  given  me  some  additional  techniques  and  a  more 
organized  way  to  help  me  think  about  how  I  influence  others.”  Instructors  also  observed  that  the 
model’s  linear  steps  provided  an  easy  way  to  explain  the  concepts  to  the  students  in  a  clear  and 
logical  fashion. 

A  third  conclusion  is  that  blended  solutions  may  provide  optimal  mix  of  training  for 
interpersonal  knowledge  and  skills.  Technology  based  learning  can  be  a  useful  way  to  provide 
foundational  concepts  and  then  test  for  comprehension.  Classroom  time  can  then  be  spent 
engaged  in  deliberate  practice  and  feedback  to  reinforce  the  concepts  taught  in  the  earlier  online 
modules.  A  blended  solution  may  be  particularly  useful  for  field  settings  where  classroom  time  is 
very  limited.  Technology-based  training  modules  may  also  be  helpful  for  National  Guard  or 
Reserve  troops  as  they  prepare  to  serve  temporarily  on  active  duty. 

A  fourth  conclusion  is  that  automated  assessments  have  promise  for  measuring  interpersonal 
and  leadership  skills  but  more  research  is  needed  to  establish  the  validity  of  these  assessments. 
For  example,  results  from  the  SJT  and  role-play  were  encouraging,  but  resource  limitations 
prevented  us  from  collecting  additional  data  to  compare  scores  on  these  measures  to  more 
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objective  assessments  of  social  awareness  and  influence  skills.  A  larger  sample  and  external 
comparisons  are  needed  to  fully  validate  these  assessments. 

A  final  conclusion  is  that  effective  technology-based  solutions  need  not  be  costly  to  develop. 
In  developing  the  WBT  Prototype,  a  “medium  tech”  approach  was  taken  rather  than  a  high  tech 
approach  to  keep  the  development  costs  low.  For  example,  rather  than  hiring  professional  actors 
and  video  production  crews  to  produce  expensive  video  for  the  SJT  and  role-play  assessment, 
still  photos  were  used  to  capture  appropriate  images  and  recorded  corresponding  audio  to 
develop  a  slide  show  that  combines  the  photos  with  voice-over  narrative.  Although  students  may 
react  more  positively  to  training  that  is  highly  entertaining,  there  is  no  evidence  to  suggest  that 
such  expense  is  needed  to  promote  learning.  In  fact,  it  has  been  argued  that  training  that  is  too 
flashy  actually  takes  away  from  the  educational  value  of  the  material  (Clark  &  Mayer,  2008). 

Next  Steps  and  Future  Directions 

The  findings  from  this  research  suggest  that  the  prototype  WBT  POI  offers  promise  and 
could  benefit  from  further  enhancements  -  either  as  a  stand-alone  training  program  or  when 
integrated  into  a  blended  solution.  Based  on  the  results  of  the  pilot  tests,  specific 
recommendations  for  enhancing  the  WBT  program  and  assessments  include: 

♦  Adding  6-8  new  learning  interactions  to  reinforce  important  concepts. 

♦  Increasing  animation  on  some  of  the  lecture  content  by  including  additional  fade-ins  and 
pop  ups  on  15-20  slides. 

♦  Adding  some  sound/narration  to  up  to  50%  of  the  slides  in  the  training  modules.  Note 
that  this  narration  would  not  simply  repeat  the  contents  of  the  slide.  Rather,  it  would 
involve  providing  a  summary  of  key  points,  narrating  some  of  the  vignettes,  and 
providing  additional  examples  and  learning  points. 

♦  Adding  some  feedback  to  the  role-play  exercise  so  that  participants  receive  a  debriefing 
on  the  Mukhtar’s  perspective.  This  would  tie  the  experience  with  the  Mukhtar  back  to  the 
content  of  the  course  and  help  improve  metacognitive  skills. 

♦  Developing  an  avatar  coach  for  use  during  the  Mukhtar  role-play  to  provide  some 
additional  coaching  and  feedback  throughout  the  exercise.  In  this  case,  the  automated 
role-play  would  be  used  primarily  as  a  learning  tool  rather  than  as  an  assessment  device. 

♦  Enhancing  the  quality  of  the  video  in  the  SJT,  this  would  entail  reshooting  the  videos  by 
taking  more  still  photos  and  re-recording  the  narration. 
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Appendix  A:  Complete  Results  from  Student  Reactions  Questionnaire  (Classroom) 


Min 

Max 

M 

SD 

1 .  This  course  helped  me  to  better  understand  the 

importance  of  social  awareness  and  influence  to  Army 
leaders. 

1 

5 

4.10 

.692 

2.  This  course  helped  me  to  leam  more  about  my  own 
strengths  and  development  needs  related  to  social 
awareness  and  influence. 

2 

5 

4.00 

.684 

3.  This  course  helped  me  to  better  understand  the  social 
awareness  and  influence  process. 

2 

5 

4.15 

.618 

4.  This  course  helped  me  to  better  recognize  and  accurately 
interpret  verbal  and  nonverbal  cues  from  others. 

2 

5 

3.96 

.713 

5.  This  course  helped  me  to  better  change  my  own  behavior 
to  more  effectively  interact  with  other  people. 

2 

5 

3.88 

.733 

6.  This  course  helped  me  to  become  a  better  influencer. 

2 

5 

3.92 

.739 

7.  Overall,  I  believe  that  what  I  learned  in  this  course  will 
help  me  to  be  a  better  leader. 

2 

5 

4.06 

.727 

8.  The  instructors  effectively  facilitated  this  course. 

3 

5 

4.56 

.542 

9.  The  course  materials  were  organized  and  easy  to  use. 

2 

5 

4.46 

.651 

10.  Overall,  I  believe  this  course  was  worthwhile. 

2 

5 

4.10 

.831 

1 1 .  I’m  planning  to  use  what  I  learned  in  this  course  in  future 
interactions  with  other  people  on  the  job. 

3 

5 

4.19 

.680 

Note:  A  =48. 
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12.  The  difficulty  level  of  the  material  presented  in  this  course  was: 


Frequency 

Valid 

Percent 

Too  complex  or  difficult 

2 

4.3% 

Too  basic  or  simple 

4 

8.5% 

Appropriate  -  neither  too  basic  nor  too  complex 

41 

87.2% 

13.  The  length  of  this  course  was: 


Frequency 

Valid 

Percent 

Too  long  for  the  amount  of  material  presented 

4 

8.5% 

Too  short  for  the  amount  of  material  presented 

1 

2.1% 

Appropriate  -  neither  too  long  nor  too  short 

42 

89.4% 

14.  What  was  the  most  useful  part  of  the  course  and  why? 

■  I  enjoyed  the  practical  exercise  role-plays  with  peers.  Very  useful. 

■  Role-playing.  It  allowed  the  class  to  practice  new  techniques  and  approaches. 

■  Influence  tactics.  Gives  others  options  and  ideas  for  use. 

■  Instructions  on  the  ways  to  influence. 

■  Final  performance  exercises  (role-plays). 

■  Practical  Exercises  and  “Role-playing” 

■  Influence  tactics 

■  Different  vocabulary  skill,  different  body  language 

■  Teaming  up  because  we  used  what  we  were  learning 

■  Knowledgeable  Instructors 

■  Going  over  the  different  types  of  influences 

■  About  the  verbal  and  nonverbal 

■  Visual  aides  because  it  gave  us  a  visual  understanding  on  what  they  were  talking  about 

■  The  scenarios  helped  build  on  skills 

■  It  was  all  good 

■  How  to  talk  to  people  to  accomplish  your  goals 

■  Negotiation  techniques 

■  Introduction  to  words  I’ve  never  heard  of.  It  makes  you  think  about  how  you  talk  to 
people. 

■  Role-playing.  Being  serious  and  interacting  with  the  material. 
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■  The  open  discussion 

■  To  be  a  better  influence 

■  Great  instructors  really  helped 

■  The  instruction  of  changing  tactics  while  trying  to  influence 

■  It  was  good  learning.  Don’t  know  if  I  would  ever  us  it!  Civilian  based  not  military. 

■  I  learned  more  about  social  awareness,  things  I  didn’t  know 

■  Performance  exercises  (role-plays)  let  the  students  execute  what  they  know 

■  Presenting  various  influencing  techniques 

■  Face  and  verbal  reactions,  understand  what  people  are  thinking 

■  All  of  it 

■  Role-playing 

■  Different  tactics! 

■  Visual  aides,  training  aides 

■  Learned  a  lot  of  things  I  didn’t  know 

■  Influence  tactics 

■  Understanding  what  tools  are  available  to  help  influence  others 

■  The  scenarios,  hands  on  training  is  always  best 

■  The  accelerated  performance  exercises  (role-plays) 

■  Identifying  different  influences 

■  Influence  model/exercises 

■  Going  over  the  different  influence  models,  explaining  what  each  one  was,  strengths  and 
weakness,  and  how  to  properly  apply  them. 

■  Identifying  the  different  influencing  tactics  and  how  to  recognize  when  to  alter  them 

■  Role-playing  scenarios  help  understands  why  this  types  of  training  is  necessary  and  how 
often  it  can  be  used 

■  The  scenarios 

■  Planning  influence  tactics  prior  to  trying  to  influence  someone 

■  The  different  scenarios  that  were  used  in  the  partner  exercises  really  helped  me 
understand  the  material  presented 

15.  What  was  the  least  useful  part  of  the  course  and  why? 

■  Umm. . .  can’t  think  of  one. 

■  Cultural  dimensions  self-assessment.  It  was  not  tied  in  very  well  with  the  rest  of  the 
course. 

■  Beginning  Power-Point  Deluge 

■  The  video  scenarios 

■  Military  to  military  influence 

■  Army  Reference,  a  person  with  no  influence  would  not  gain  skills  from  a  single  day 
course 

■  Nothing 

■  It  was  all  good. 

■  The  video 

■  Last  slide  show  -  it  was  lame. 

■  The  slide  video 

■  Learning  body  language  of  people 
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■  Study  Guide  is  all  that’s  needed 

■  Most  of  us  already  use  this  information 

■  All  good  information. 

■  Everything  was  good. 

■  The  movies 

■  Nothing  really 

■  Part  about  facial  expressions.  I  think  it  is  common  knowledge?! 

■  The  last  video.  I  felt  it  was  beneath  the  audience  level  of  experience.  Been  through  this 
scenario  before. 

■  Much  of  the  book  information  not  really  talked  about  (in  class).  Maybe  useful  later. 

■  Video  exercise  was  too  easy 

■  Communication  sergeant  -  most  of  us  already  know  most 

■  The  repetition  of  slides 

16.  Other  comments/suggestions: 

■  Kudos  to  Erin  and  Cody!  Have  a  safe  flight  back. 

■  Need  to  have  free  coffee  available  when  instructing  officers 

■  Remove  from  context  of  Army,  too  many  variables  (or  at  least  acknowledge  the 
variables)  to  gain  a  pure  understanding  of  social  influences. 

■  Make  it  more  days 

■  The  way  they  talked  was  a  little  too  technical  for  some  understanding.  Could  understand, 
but  just  too  technical. 

■  Good  training 

■  Probably  better  for  officers 

■  Better  videos 

■  Great  job. 

■  Use  NCO  videos  for  NCOs  and  officer  videos  for  officers 

■  Class  was  too  long 

■  More  practice  on  applying  new  techniques  of  influence 

■  Good  course 

■  Well  planned  and  executed  class  and  good  flow  of  material  presented 

■  The  course  was  really  good 

■  I  think  the  course  could  be  streamlined  to  fit  a  six  hour  block  of  instruction 

■  Excellent  course 

■  Great  instructors  very  well  explained  and  taught 

■  More  audience  involvement  and  less  PowerPoint  slides.  The  military  see  these  quite  often 

■  Good  course,  well  taught  and  well  paced.  Very  informative 

■  Good  workshop  and  outstanding  presentation 
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Appendix  B:  Complete  Results  from  Student  Reactions  Questionnaire  (WBT) 


Min 

Max 

M 

SD 

1 .  This  course  helped  me  to  better  understand  the  importance 
of  social  awareness  and  influence  to  Army  leaders. 

2 

5 

4.09 

.689 

2.  This  course  helped  me  to  learn  more  about  my  own 
strengths  and  weaknesses  related  to  social  awareness  and 
influence. 

2 

5 

3.91 

.818 

3.  This  course  helped  me  to  better  understand  the  social 
awareness  and  influence  process. 

2 

5 

4.28 

.683 

4.  This  course  helped  me  to  better  recognize  and  accurately 
interpret  verbal  and  nonverbal  cues  from  others. 

2 

5 

3.84 

.860 

5.  This  course  taught  me  how  to  change  my  own  behavior  to 
more  effectively  interact  with  other  people. 

2 

5 

3.72 

.851 

6.  This  course  helped  me  to  become  a  better  influencer. 

2 

5 

3.78 

.906 

7.  Overall,  I  believe  that  what  I  learned  in  this  course  will  help 
me  to  be  a  better  leader. 

2 

5 

4.13 

.707 

8.  The  web-based  interface  was  effective  in  delivering  the 
material  in  this  course. 

1 

5 

3.75 

1.078 

9.  Instructions  were  easy  to  understand  and  follow. 

3 

5 

4.25 

.508 

10.  The  overall  look  and  feel  of  the  course  was  engaging. 

1 

5 

3.87 

.907 

1 1 .  Navigating  through  the  course  was  easy. 

1 

5 

4.16 

.920 

12.  The  course  was  easily  accessible. 

1 

5 

4.25 

.842 

13.  The  practice  questions  and  activities  were  relevant  and 
reinforced  learning. 

3 

5 

4.16 

.515 

14.  Overall,  I  believe  this  course  was  worthwhile. 

2 

5 

4.12 

.751 

15.  I’m  planning  to  use  what  I  learned  in  this  course  in  future 
interactions  with  other  people  on  the  job. 

2 

5 

4.19 

.821 

16.  This  course  helped  me  to  better  understand  the  importance 
of  social  awareness  and  influence  to  Army  leaders. 

2 

5 

3.97 

.861 

Note:  N  =  32. 


17.  The  difficulty  level  of  the  material  presented  in  this  course  was: 


Frequency 

Valid 

Percent 

Too  complex  or  difficult 

0 

0 

Too  basic  or  simple 

4 

12.5% 

33 


Appropriate  -  neither  too  basic  nor  too  complex 


28 


87.5% 


18.  The  length  of  this  course  was: 


Frequency 

Valid 

Percent 

Too  long  for  the  amount  of  material  presented 

4 

12.5% 

Too  short  for  the  amount  of  material  presented 

3 

9.4% 

Appropriate  -  neither  too  long  nor  too  short 

25 

78.1% 

19.  If  you  were  going  to  take  a  course  like  this  in  the  future,  which  format  would  you  prefer? 


Frequency 

Valid 

Percent 

Online 

11 

34.4% 

Traditional  Classroom  Setting 

16 

50% 

No  Preference 

5 

15.6% 

20.  What  did  you  like  most  about  the  course  and  why? 

■  The  real  examples 

■  Scenarios-  allowed  me  to  understand  what  I  read 

■  The  Scenarios  were  real-world  and  applicable 

■  Ability  to  check  for  correct  answers  because  it  reinforced  leaning  objectives 

■  Good  content.  Good  organization  of  material  and  it  flowed 

■  The  info 

■  Instructor  readily  available  for  computer  glitches 

■  Simple 

■  I  liked  the  fact  that  this  is  part  of  the  coursework.  This  is  good  training  for  every  military 
leader. 

■  The  scenario 

■  The  subject  matter  because  it  will  help  me  in  many  ways  other  than  in  my  duties  for  the 
Army. 

■  Check  on  learning  and  clear  learning  objectives 

■  Summary 

■  The  self  checks  thru  the  lecture 

■  Relevant  infonnation  that  I  can  use  for  my  assignment. 

■  It  gave  good  examples  and  the  material  was  worthwhile 

■  I  liked  the  specific  influence  tactics  and  the  situations  they  would  and  would  not  be 
useful  in. 

■  It  is  a  great  basic  course  to  prepare  an  individual  for  a  classroom  setting  course. 

■  How  the  checks  on  learning  supported  the  material 
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■  Great  information  and  useful/appropriate  real-life  examples. 

■  Multiple  checks  on  learning.  I  was  efficiently  challenged  and  not  too  overwhelmed  by  the 
information  covered. 

■  Self  paced 

■  Short,  to  the  point 

■  The  information  on  the  Middle  East  Culture. 

■  The  relevance  to  current  operations.  I  especially  enjoyed  the  interaction  exercises  with 
tribal  leaders. 

■  I  liked  the  vignettes,  they  seemed  realistic  and  facilitated  thinking  outside  of  the  box 
(chain  of  command,  I  do  it  because  it  is  an  order). 

2 1 .  What  did  you  like  least  about  the  course  and  why? 

■  On-line  course  method 

■  Undefined 

■  It  was  difficult  to  be  motivated  to  continue  with  this  course  due  to  the  8  times  I  had  to 
restart  because  of  technical  difficulties. 

■  Would  prefer  some  audio  to  reinforce  key  topics 

■  You  can’t  teach  social  skills  thru  a  computer!  this  would  be  a  great  prerequisite  course  for 
an  actual  live  person  teaching  the  subject  in  depth 

■  It  is  hard  to  study  on  the  computer  for  hours 

■  Taking  it  after  lunch,  cramped  classroom.  Computer  glitches 

■  Undefined 

■  This  material  seemed  too  basic.  There  is  a  great  deal  of  in-depth  work  that's  gone  on  in 
influential  leadership.  Practical  exercises  and  group  dynamic  is  very  effective  in  this 
learning. 

■  N/A 

■  The  online  feel  with  no  sound  and  no  discussion  style  format  in  the  lessons. 

■  Not  enough  examples  or  scenarios 

■  Have  an  audio  option. 

■  Too  boring 

■  Reading  body  language  was  too  basic . we  understand  when  someone  is  bad,  sad,  happy 

,  etc 

■  I  think  distinguishing  between  influence/manipulation  and  greater  discussion  in  ethics 
would  be  useful. 

■  Some  of  the  slides  skip  and  the  majority  are  slow  to  react  on  a  high  bandwidth 
connection.  This  means  that  it  would  undoubtedly  take  up  more  time  for  my  Soldiers  if 
conducted  at  some  of  their  homes. 

■  Lack  of  audio  content  to  reinforce  the  written  text 

■  NA 

■  It  could  have  been  more  interactive.  It  became  mundane  after  a  while.  The  examples  were 
exciting  and  seemed  interactive  to  the  point  where  you  needed  to  adjust  your  approach  to 
get  positive  results. 

■  Distractions  and  time  pressured  to  accomplish.  It  no  longer  become  constructive  to  learn 
but  to  pressure  to  complete  because  others  of  the  team  members  completed  earlier  and 
team  member  who  completed  earlier  making  negative  comments  why  it  is  taking 

■  No  real  life  practical  exercise 
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■  The  self  evaluation 

■  Initial  startup  of  online  training  is  always  problematic,  this  format  seemed  very  easy 
though... not  many  settings  to  check  for  the  system  to  operate  properly. 

22.  Additional  comments. 

■  The  course  flowed  smoothly 

■  This  course  needs  to  continue  to  be  tested  so  that  more  bugs  can  be  worked  out. 

■  None 

■  Would  like  to  have  heard  audio  in  the  notes.  Mover  video  examples  of  topics  discussed. 

■  There  are  numerous  sales  and  leadership  training  companies  that  conduct  similar  training. 
Reviewing  their  curriculum  might  be  very  helpful. 

■  The  use  of  "her"  or  "she"  is  referenced  several  times  to  indicate  a  response  or  where  the 
idea  originated.  Might  use  "their"  or  "them"  or  "his/her"  or  "he/she"  were  it  may  apply. 

■  Need  some  listening  between  reading . when  reading  so  much  material  it  is  easy  to  get 

distracted  and  lose  focus 

■  Enjoyed 

■  This  course  is  good,  but  should  be  an  additional  preparation  not  a  replacement  for  the 
traditional  "in  house"  or  class  room  environment. 

■  Long  to  complete.  I  wish  I  had  the  time  to  absorb  the  materials  longer  time.  I  believe  the 
materials  are  valuable  in  my  success  or  completion  of  my  mission. 

■  Practice! 

■  Overall  a  good  course. 
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